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mCUTIVB SUMKARY 



B6A surveyed -ill States and territories in May and June, 1985, and 
performed the analysis with funds from the Hational Commission for Employment 
Policy. Responses from 49 States are included in the survey unless otherwise 
indicated. Given the Staters role in implementing jtPA performance standards 
a variety of policies and approaches are being taken. The greatest concern 
the States expressed was in the need for consistently defined data to 
establish and adjust standards. The survey revealed that, overall. State and 
local program operators have made significant progress in ioq)lementing a 
meaningful performance standards system but that there a number of areas where 
Federal direction is still needed. Major findings of the assessment were in 
five areas: 

A<HttaHw^ t of standards- to Account for Local Conditions 

o For py 85, forty (8») of the forty-nine States responding 
indicated that they planned to use the Employment and 
Training Administration (ETA) adjustment methodology 
(known as **the model**) to establish local performance 
standards. 

o Of the forty- three States that use the national adjustment 
methodology for at least one performance measure, 
thirty-eight (86%) recalculate the standards at the end of 
the year based on actual JTPA Annual Status Report (JASR) 
data. 

o Thirty-six of the forty States (90%) that use the model to 
establish SDA standards responded that its ability to be 
applied objectively to SDAs was a strength. 

o The greatest cause for concern among States that use the 
model is the lack of uniform data definitions. 

Use of 6% Funds 

o The percentage of funds used for incentive awards 
generally remained stable or increased between PY 84 and 
py 85. PY 85 will be the first year in which States must 
make incentive awards for exceeding performance standards. 

o In P¥ 84 the proportion of funds used for incentive amrds 
ranged from zero to 100%. One-fifth of the States 
responding used no 6% funds for incentive a%#ards in PY 84. 

- Six of these States (13%) used 100% of the 

funds for technical assistance. , 

- Twenty-one States (46%) used a portion of the 
6% funds for service to the hard-to-serve. The 
range of funds allocated for serving the 
hard-to-serve was from 2% to 10% in PY 84. 
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o In PY 85 nine states (20%) responded that they had not 
decided about the use of 6% funds. 



- The number of States planning to use a portion 
of the 6% funds for serving the hard-to-serve 
declined from twenty-one to fifteen in PY 35. 

- Thirty-five states (76%) anticipate using a 
portion of the 6X funds for technical 
assistance in PY 85, a decline from forty-three 
(93%) who planned to do so in PY 84. 

Incentive Policies 

o Of the 25 States included in the analysis of incentive 
policies t tae most common type of eligibility criterion 
for an incentive award is the requirement that a 
particular number of standards be met or exceeded. 

o The most common distribution calculation used by States in 
determining the amoiUiC of the incentive award is to adjust 
the amount of the potential award by the size of the 
entity's Title II-A allocation. 

o Twelve States (46%) have policies requiring that standards 
be exceeded before incentive funds could be awarded. 

o Four States require that a certain percentage of the Title 
II-A 78% funds be expended before a SOA could be eligible 
for an incentive award. 

Youth Employment Competencies 

o 85% of the States currently have a reporting system for 
the attainment of youth employment competencies. By the 
middle of calendar year 1986 all but one State surveyed 
would have such a system. TWo out of three States either 
use one reporting category or plan to use one — attainment 
of youth eoq>loyment competency — for reporting from SDAs, 
while remaining States use multiple reporting categories. 

o Thirty percent of the respondents were not sure where the 
attainment of a youth eiq)loyment competency %ras reported 
on the JTPA Annual Status Report. 

o 60% of the SDAs in the forty-nine States responding to 
the survey have youth entployment competency systems 
actively serving youth. States reported that an 
additional 114 SDAs will have operational systems by July 
It 1985 t which would bring the percentage to 77%. 



o Pre-employment conq)etencies are still the most conjaon 
categories of competencies offered, with work maturity a 
close seconds 

o The majority of states that had operational youth 
employment competencies systems had no additional 
administrative requirements that must be fulfilled before 
the attainment of youth esq>loyment conqpetencies could be 
reported by an SDA. The additional criteria used in some 
Stated are administrative in nature and do not appear to 
encroach upon PIC discretion. 

o Of forty-six States responding, only twelve (26%) stated 
that they adjusted either of the two youth standards in PY 
84 for the presence or absence of youth ^iq>loyment 
competencies at the local level. For PY 85, nineteen 
States (41%) indicated that they were either planning to 
adjust for the status of youth employment competencies or 
were unsure at this time. 

Post-ProKram Follow-up 

o 80% of the States responding will have operational 
follow-up systems in place by the beginning of py 86. 

- Twenty-eight states currently have follow-up 
systems in place and collecting data. Of the 
States responding , eleven States ( 22%) have 
been doing some follow-up since the inception 
of JTPA or before. 

- 57% of the 28 States that have operational 
follow-up systems require SDA-leveJ follow-up, 
with participant follow-up being the most 
ccmmonly used approach. 

o The size of the JTPA aJ.location does not appear to be n 
strong factor in whether States have in^lemented a 
follow-up system. 

o The vast majority of States (77%) contact the participant 
and/or the employer at thirteen weeks after the 
participant terminates. 

o Over half the States responded that all terminees (not 
just those placed) were included in the post-program 
system. 

o 86% of the States follow-up on all age groups within the 
terminee population (adults, youth, and adult welfare 
recipients). 



IMTRODUCTIOH 



Thm Vational Governors* Association (HGA) conducted aa assessment of the 
ttatus of perfonsance standards used in the Job Training Partnership Act 
(J1PA) Title II-A and Title III programs in April and Hay, 1985. The analysis 
of the assessment was funded by the Vational Commission for Employment Policy 
(HCSP). A questionnaire i#as mailed to each State, the District of Columbia, 
and the Territories on April 25, 1985, and follow-up telephone calls were made 
to States during Kay to obtain the information. 

PUBPOSB 

The purpose of this assessment is to obtain information on the status of 
performance standards as JTPA approaches establishment of FY 86 performance 
standards. FY 86 will be the first opportunity since the inception of the 
initial standards to make changes in the performance standards system because 
the Act at 106(d)(4)(A) states that the Secretary may not modify the standards 
more often than once every two years. Consistent with this statement, the 
Eoq>loyment and Training Administration (ETA) issued a notice in the Federal 
Register on Oecesiber 4, 1984, indicating that it would not change the 
numerical values for the national standards for FY 85. Also, because of the 
interest throughout the employment and training system in establishing 
post-program performance standards, HGA decided it was important to provide 
information to policymakers on the current status of performance standards in 
the States to better inform the direction of any changes for FY 86. Finally, 
VGA %fas able to use this opporttmity to consolidate requests for information 
in this area with two other organizations with varying responsibilities 
related to performance standards. The General Accounting Office (GAG) was 
particularly interested in the youth eo^loyment competency information as part 
of a more comprehensive study in this area in which they were engaged. 

The Vational Commission for Biqiloyment Policy will use the analysis of 
this status report to assist them in their statutory mandate to advise the 
Secretary of Labor on the development and iaqplementation of JTPA performance 
standards. VGA will also use the results of this assessment to guide our 
technical assistance efforts in the performance standards area and in advising 
the Department of Labor (OOL) on changes needed in FY 86 performance 
standards. 

BACKGROUVD 

The Act, at Section 106, directs the Secretary of Labor to establish 
performance standards to determine idiether the basic goals of the program are 
achieved. The Act suggests certain measures to determine whether the stated 
goals of increasing enQ>loyment and earnings and reducing welfare dependency 



are being met. The Act gives the Governor the responsibility of varying the 
standards established by the Secretary based upon specific economic, 
geographic, and demoeraphic factors in the State and in service delivery areas 
within the State, the characteristics of the population to be sexrved, and the 
type of service to be provided 

The Employment and Training Administration issued performance standards 
for PY 84 on January 31, 1984 in Performance Standards Issuance Number I-PY 
84. This directive contained information on the seven measures used to gauge 
program success, the national numeric standards associated with each measure, 
parameters within which the Governor could vary the standards, and other 
related information. The performance measures and the Secretary's national 
standards for PY 84 were: 

Adult 

Entered En^loyment Rate — The number of adults who entered 
eflq>loyment at termination as a percentage of adults who 
terminated : 55%. 

Cost per Entered Employment — Total expenditures for adults divided 
by the number of adults who terminated: $5,704. 

Average Wage at Placement — Average wage for all adults who entered 
employment at the time of termination: $4.91. 

Welfare Entered Employment — The number of adult welfare recipients 
who entered employment at termination as a percentage of the 
number of adult welfare recipiences who texrminated: 39%. 

Youth 

Entered Employment Rate — The number of youth who entered 
enqployment at termination as a percentage of the number of youth 
who terminated: 41%. 

Positive Termination Rate — ^The number of youth who had a positive 
termination (i.e., at termination, the youth had either entered 
unsubsidized employment; or had mot one of the youth employability 
enhancement definitions; or had attained youth employment 
coiiq>etencies recognized by the PIC) as a percentage of the total 
youth who terminated: 82%. 

Cost per Positive Termination — Total expenditures for youth 
divided by the number of youth who had a positive termination (see 
above for definition of positive termination): $4,900. 
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These same measures and numeric standards were also used for PY 85. 

ETA also issued a Technical Assistance Guide (TAG) to perfonnance standards 
which contained information on how States could use the optional sta adjustment 
methodology to adjust for local conditions. This adjustment methodology 
consisted of adjustments for local factors shown to influence performance and 
was developed using a statistical technique called multiple regression 
analysis. This adjustment approach is sometimes referred to as **the model" by 
practitioners. 

HKSPOBDKMTS 

Surveys were mailed to all JTPA State Liaisons. Responses were received 
from forty-eight States and the District of Columbia for a response rate of 
86%. Respondents were primarily those managers or technicians responsible for 
performance standards in each State. In many cases there were multiple 
respondents since the questionnaire solicited information on a variety of 
programs. This report is based on these voluntaxry responses. The number of 
actual responses may vary for a given question and will be so indicated for 
each section or question. The survey instrument had four parts: 



SECTION I. 



GENERAL 



The questions in this section referred to both planned and 
actual performance standards act ivx ties for ?Y 84 (7/84-6/85) 
and PY 85 (7/85-6/86) and deal with the current seven 
performance standards, adjustments made to these standards 
(particularly for the four adult standards), and the use of 6% 
set-aside funds for incentive awards, technical assistance, or 
incentives for the hard-to-serve. 



Section II. 



Youth Bnq^loyment Con^etency and Termination Measures 



The questions in this section related to the status of youth 
employment competency systems within a State, reporting of 
conqpetency attainment, adjustments made to account for the 
presence or absence of a youth competency system, and the 
weights given to youth performance measures in making incentive 
awards. 



Section III. Post-Program Standards 



This section was designed to obtain information on both the 
current status of follow-up systems or plans for such systems, 
the methods used for follow-up, the information obtained, and 
whether performance measures and numeric standards have been 
established using post-program iata. 



section IV* 



Additional Performance Standards 



This section focused on whether additional performance 
standards were established for Title II-A for PY 84 or were 
planned for PY 85 and whether standards were established for 
older worker projects (3% projects ) , State education 
coordination grants (8% projects) , or other target groups • 
Questions w^re also included about the establishment of Title 
III standards for both PY 84 and PY 35. 
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PIMDIgGS; 

SBCTIOH I. GENERAL 

How Local Service Delivery Area (SPA) Standards Were Established 

Of the forty-nine respondents to this section, forty-eight responded 
resarding FY 84 and thirty-nine of those responding (81%) used the ETA 
regression model in FY 84 to adjust the national performance- For FY 85, 
forty-nine States responded to this question and forty (82%) responded that 
they planned to use the ETA adjustment methodology to account for local 
conditions. Use of the regression model as a starting point to adjust the 
Secretary's seven standards has remained relatively stable since the 
transition year (TY 84) and may have increased slightly since not as many 
States adjusted the national performance standards during FY 84. For 
information on which specific States use various methods see Table II. 



Table I: Summary Of states 
Responses 





PY 84 


PY 85 


Used the ETA adjustment methodology 


39(81%) 


40(82%) 


Used some other method 


4 (8%) 


4 (8%) 


Used the national standards only 


1 (2%) 


2 (4%) 


Used a combination of the national 






standards and ETA adjustment model 


4 (8%) 


3 (6%) 


Total Responding 


48 


49 



Of the States which used a combination of the national standards and 
the ETA adjustment methodology in FY 84, four used the ETA adjustment 
model to adjust only the average wage at placement standard. 
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Type of Adjustments Made Beyond the National Adjustment Methodology (for Adult 



Standards) 



States that used the DOL adjustmenl :nodel were also asked about the types 
of adjustments made (or to be made) beyond the national adjustment model. 
States adjust for a variety of reasons, as the Act envisioned, but the most 
common reason slven was for extreme values for client characteristics in the 
model. 

Interpretation of the responses must be tempered by some of the additional 
information provided by States • In many cases , States responded that SDAs 
must request an adjustment beyond the model, and they had not received such 
requests. It is not possible to know if any of the ten States that responded 
that no additional adjustment (beyond the model-predicted number) have been 
made would have been willing to make such adjustments if they had been 
requested. Several States that listed specific types of adjustments beyond 
the national model indicated that their responses reflected State policy on 
allowable adjustments but that no adjustments were made in PY 84. (^nerally. 
States noted that the same adjustments would apply for PY 85. 

Of the thirty-nine States that used the model in PY 84, the responses were 
as follows: 



Table III: Reasons for Adjustments 

Number of States 



Bx'treme values for client characteristics 13 

Services to groups not in the model 7 

Extreme values fo.r unemployment rates 9 

(or changes not reflected in the rate) 

Difference in average wage in the area 8 

Lack of training facilities 2 

Type of service (OJT, classroom training job 2 

search assistance) 

Other 14 

None 10 



Note: Multiple responses were encouraged 



Of the fourteen States that indicated that they use ''other'* adjustments 
than those listed there were a variety of approaches. The types of 
adjustments included: use of the tolerance range alone; adjustments for 
State-determined minimum and maximum ranges based on State data to enlarge the 
tolerance range; a 10% productivity increase to the Entered Employment Rate 
standard if public transportation was available within an entire SDA; 
productivity adjustments to all adult standards based on past experience ; 
lowering the cost per entered employment for PY 85 outside the tolerance range 



based on actual transition year and PY 84 data; neutralizing the negative 
effect of ''single head of household** in the cost per ente-^ed employment 
standard for PY 84; reducing the model-produced average wage rate by a certain 
percentage. 

There was no clear pattern as to which States actually made adjustments 
beyond the regression model. Single-State SDAs used adjustments beyond the 
model as frequently as States with multiple SDAs, perhaps because it gave them 
a management assessment tool for their local planning areas. 

The national performance standards include a productivity improvement 
factor for all measures except the cost measures. The ETA adjustment model 
does not include a built-in productivity improvement factor. Only two States 
indicated that they included some productivity improvement factor. 

Recalculation of Performance Standards at the End of the Year 

Of the forty-three States that use the national adjustment methodology for 
at least one measure in PY 84, thirty-seven (86%) recalculated the standards 
at the end of the year based on actual JTPA Annual Status Report (JASR) data 
as was envisioned in the use of the model. 2/ In PY 85, thirty-nine States 
(91%) indicated that they would recalculate standards at the end of the year. 
The questionnaire did not reveal why two States that did not recalculate in PY 
84 plan to do so in PY 85. Of those States that do not recalculate the 
standards, the survey does not reveal if this is because of a misunderstanding 
of the principles of the modeling approa ,h or due to a deliberate policy 
decision. 

Greatest Strengths and Weaknesses of the POL Adjustment Methodology 

States were asked to give three responses to questions concerning their 
perceptions of the strengths and weaknesses of the DOL adjustment methodology. 

The responses on the strengths of the model ranked as follows with the 
number of responses to each choice in parenthesis: 

It treats all SDAs objectively (36) 

It does not base performance on whether SDAs (21) 
planned correctly, but rather holds the SDA 
accountable for whom they serve 

It reflects system differences while providing (20) 
a standardized system which all States can use 
to gauge performance 
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It provides States with the flexibility to target (16) 
service to certain groups (such as women, welfare 
recipients, or dropouts) without penalizing the SDAs 
for lower performance 

It quantiifies most local factors which affect (12) 
performance in my State 

It allows SDAs to serve those most in need without Cll) 
being penalized 

It is easy to understand and use (4) 
Other (4) 
The ''other** responses included: 
0 Truly acts as an incentive to perform 

o With small staff standards can be adjusted using a sound method 
approved by the Secretary of Labor 

o Allows ' States to target and award performance on statewide goals 
which may vary from national goals 

We also asked each State about the greatest weaknesses of the DOL 
adjustment methodology. Responses ranked as follows: 

o It is hard to understand and explain to (29) 
SDAs and PICs 

o The national data base upon which the (21) 
model is based doesn't reflect our State *s 
conditions 

0 The lack of uniformity in data definitions (19) 
has resulted in unreliable weights for 
local factors 

o Other (15) 

o Hot enough local factors are in the model (13) 

0 It is too complex and time consuming to (5) 
administer 
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The **oth8r'* responses include: 



o Concern about the validity of the data 
base (either CETA for the PY 84 model or 
JTPA for PY 85) t and/or the uniformity of data 
definitions (six responses) 

o The absence of youth employment competencies in the model 
dictates program mix for youth 

o Weights are too low to act as an incentive to serve certain 
groups 

o Reversing values of coefficients for same factor within 2-year 
planning cycle is a problem 

o The model doesn't reflect the rapidity of economic change and 
has a regional bias 

o The standards are too cost oriented and drive the system 
towards short-term results 

The responses to both these questions confirm the general positive 
perceptions of the OOL adjustment methodology. The shift away from 
traditional **planned vs. actual** analysis of performance appears to be 
accepted, and 36 of the 40 States that use the model to establish SOA 
standards responded that its ability to be applied objectively to SOAs was a 
strength. While States are concerned about understanding and explaining this 
methodology, it is not viewed as being too difficult to actually administer. 
There is very strong su^;;ort for the objective approach for establishing local 
performance standards as evidenced by the 36 responses to this question. The 
earlier responses about adjustments beyond the DOL model seem to Indicate that 
States are searching for straightforward ways to further refine the establish- 
ment of standards and ways to quantify additional adjustment within the 
limitations of available staff time. 

A great cause for concern among the responses is the emphasis on the lack 
of uniform data definitions (19 responses) and the repeated mention of data 
problems .In the additional responses. As many States pointed out, the model 
is only as good as the data used to produce it. 

Use of 6% Funds for Incentive Awards, Technical Assistance > and Incentives for 
Serving the Hard-to-Serve> 

Section 202 (b)(3)(B) of the Act specifies that 6% of the allotment of 
each State shall be used by the Governor **to provide incentive grants for 
programs exceeding their performance standards, including incentives for 
serving hard~to-serve individuals.** The Ac further states that incentives 
must be distributed based on the degree by which the SOA exceeds its 
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performance standards. Funds not needed for incentive grants are to be used 
for technical assistance to SDAs no^ qualifying for incentive awards. 3/ 
This section, then, deals with the use of 6% funds for incentive awards, 
technical assistance, and incentives for serving the hard-to-serve. 

It is always difficult to assign expenditure of 6% funds to a particular 
program year since States have three years to expend the funds. Incentive 
awards for performance are necessarily viewed as ** retrospective"* activities, 
e.g., it is acceptable to reward PY 84 performance with FY 85 6% funds; and 
States may always use the current year's incentive funds to reward a prior 
year's performance. However, this does not have to be the case with all uses 
of the 6% funds. Incentives to serve the hard-to-serve may be either 
prospective (through RFPs); contemporaneous; or retrospective (FY 84 funds 
carried over into PY 85 but used to reward PY 84 performance). Finally, 
whether technical assistance can be prospective, (preventative technical 
assistance for all SDAs) , or retrospective (only those who have actually 
failed to meet standards at the end of the year), also varies from State to 
State. Therefore, much care must be taken in analyzing the responses on the 
division of 6% funds. 

In PY 84 the proportion of funds used for incentive awards ranged from 
^ero to 100%. Nine (20% of the 46 States responding) States planned to use no 
6% funds for incentive awards in PY 84. Only two of these were single-State 
SDAs. Six States (13%) used 100% of the funds for technical assistance. 
Twenty-one States (46%) used a portion of the 6% funds for service to the 
hard-to-serve. Eleven States (22% of the 49 States) responded that they would 
use means other than separate performance standards to provide incentives for 
serving the hard-to-serve. Such approaches included: Requests-for-Proposals 
(RFPs) for service to special target groups, challenge grants to SDAs to serve 
certain groups, requiring SDAs to choose target groups from their eligible 
population to serve. States did indicate that they would establish 
performance standards to assure equitable service for specific target groups 
and include such standards in their incentive awards system in some cases. 
The range of funds allocated for serving the hard-to-iserve was from 2% to 10% 
in PY 84. Two States, Hawaii and Virginia, used the majority of PY 84 6% 
funds for the hard*to-serve . 

In PY 85 nine States (20%) responded that they had not decided abo.it the 
use of 6% funds (**don*t know"* in the table) in PY 85, and one simply did not 
respond to this question. Two States explicitly stated that 100% of 6% monies 
would be used for incentive awards. The number of States planning to use a 
portion of the 6% funds for serving the hard-to-serve declined trom twenty-one 
to fifteen in PY 85* Three Stc.t^:» t^^lanned to increased the portion of funds 
used to serve the hard-to-serve; the lur^jority of States planned to keep this 
figure stable. Thirty-five States (76%) plannr,^ to use some of the funds for 
technical assistance in PY 85, a decline from forty-three (93%) who planned 
to do so in PY 84. Again, this finding must be interpreted with caution and 
does not necessarily presume a decline in the technical assistance provided 
since States have no way of knowing up front how much tochnical assistance may 
be needed for SD/s who either fail to meet standards or appear to be in danger 
of failing to meet standards during the year. The technical assistance 
figures may test be used as a planning figure, which States may change as 
circtmstances thange* 

11 
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Fifteen States (33% of those responding) expected to keep the same 
division of 6X fimds among the three categories for PY 34 and FY 85. Where 
changes were anticipated in the division of fimds* increases In the percentage 
reserved for incentive awards occurred in fourteen States. Nine States made 
no incentive amrds for performance during the transition year and used 6% 
funds for other purposes. Increases in percentage of funds reserved for 
technical assistance were planned in six cases. Table IV contains the 
specific responses to this question. 

Interpretation of information from this question must be made with great 
care* particularly in making year-to-year conq>arisons . Eight States answered 
that they were not sure (*^don*t know*') of the break down between activities 
for PY 85 wtdch may be due to a variety of reasons. One State said they were 
still using transition year funds and were not sure when PY 85 ftmds would be 
used. Other States' responses may be imcertain because plans had not been 
completed for PY 85 at the time of the survey. In fact, States frequently 
indicated that their PY 85 responses were tentative. 

Percentage of funds used for incentive awards generally remained stable or 
increased between PY 84 and PY 85. This is as expected since PY 85 will be 
the first year in which States mu' c make incentive awards for exceeding 
performance standards. During PY 85 » States will be making incentive awards 
based on PY 84 actual performance. 

One of the more interesting aspects of this analysis is the behavior of 
single-State SDAs. Even though no one was required to make incentive awards 
in PY 84 for transition year performance, many single-State SDAs apparently 
chose to do so. Some of these single-State SDAs are planning to make awards 
to substate planning areas or to specific contractors. The single-State SDAs 
have generally implemented a meaningful performance standards system including 
their use of 6% funds. 
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Type Of Technical Assistance Activities Provided With 6% Funds 

States were asked to indicate multiple uses of the 6% funds in the 
technical assistance area. Responses to this section must be interpreted with 
care. The following suosnary table indicates the incidence of various uses of 
technical assistance funds throughout the States. It does not, however, shed 
light on the proportion of 6% resources devoted to any one activity within the 
State. Thus although six States plan to use 6% funds for MIS development, we 
do not know the level of investment this represents. 



Table V: USB OF TBCHVICAL ASSIST^CB FUNDS 

No, of States Responding 



Activity 



Assist SDAs who fail to meet 
standards at end of year 

Assist SDAs likely to 
fail standards based on 
current performance 
(preventative T.A.) 

Establish Misi^ 

Maintain Misi^ 

Other 

Establish follow-up system 
N.A 



PY 84 

29(59%) 

27(55%) 



23(47%) 
17(35%) 
16(33%) 
11(22%) 
3(6%) 



PY 85 
33(67%) 

34(69%) 



6(12%) 

15(31%) 

13(27%) 

10(20%) 

3(6%) 



l/'*Establish MIS** was intended to mean development and start-up costs of new 
systems. *Kaintain MIS** was intended to mean ongoing costs. 



Responses varied considerably from PY 84 to PY 85 particularly in the area 
of establishing management information systems. While twenty-three States 
used a portion of the 6% ftmds for this activity in PY 84, only six States 
planned to do so in PY 85. This drop in the number of States who use 6% funds 
to establish management information systems may mean that the majority of 
States have completed major acquisition of hardware and software. 

Fifteen Staces (31%) indicated that they would use 6% ftaids for some MIS 
activity in both PY 84 and PY 85. This may mean a continuance of on-going MIS 
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projects or the addition of a nsw HIS component. Fifteen States (31X) 
indicated that they plan to use 6% ftinds to maintain their HIS at least at 
some minimal level in PY 85 • 

Some States indicated that they will use a portion of the 61 funds to 
establish follow«-up systems. Of the eleven States that indicated this use of 
funds for PY 84 and/or PY 85, seven already have some type of follow-up system 
(according to responses in Section III of the survey). Several of the States 
which hava been most active in developing follow-up systems are using a 
portion of the 6% funds to pay for their systems. Two of the States that 
\ndicated they would use 6% funds for follow-up systems have no such system 
but plan to develop them before the beginning of PY 86 (according to responses 
in Section III). Two States did not indicate a starting date for follow-up. 
These responses shed some limited light on how States either are currently 
paying for follow-up system or plan to do so. 

Responses to this question also indicate that half of the States are using 
technical assistance fur^.s either in the preventative technical assistance 
mode, e.g. to fund training or other activities designed to improve knowledge 
and management abilities before problems result in failure to meet performance 
standards at the end of the year, or to assist SDAs which actually do end up 
failing to meet standards. The number of States planning preventative 
technical assistance increased from PY 84 to PY 85 in about the same 
proportion as the number of States that plan to increase assistance to SOAs 
who fail to meet standards at the end of the year. Responses on **other** uses 
of technical assistance fundr include'!: general Statewide training (8 
responses), provide labor market information (3), special projects such as an 
exemplary youth projects (2), peer-to-peer technical assistance (1), formula 
funding a portion of funds to SDAs for "administrative enhancements** (1), 
employer outreach and marketing (1). 

Incentive Policies 

The Act provides at 202(b)(3)(B) that 6% of the funds allocated to a State 
shall be reserved "^to provide incentive grants for programs exceeding 
performance standards....** States have been in the process of developing 
policies that define when an administrative entity is eligible for an 
incentive award and how such awards will be calculated. The Department of 
Labor has issued no instructions or guidelines in this area except to state 
that **when the Governor establishes a system for awardinjs incentives, the 
system must include the Title II-A standards. While the system may not 
necessarily require that an SDA exceed all of the standards to be eligible 
for incentive fimds, the Governor may not disregard any of the seven measures 
in establishing the incentive system"*. 4/ in the absence of DOL policy in 
this area. States that have established additional performance standards may 
include them in their incentive system. 
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Twenty-^ight States furnished NGA with a copy of their incentive 
policies. Two of these States are not included in this analysis since the 
policies sent were still in draft form. The policies used in the analysis are 
primarily for PY 84 although States did not always indicate the time period 
applicable for the information. Incentive policies are very dynamic, and a 
number of States have loade changes to these policies since they were sent to 
NGA. 

The policies are of varying degrees of conq>lexity with most having a 
number of conq^onents which must be included before the amount of an incentive 
award can be determined. There are no clear patterns to the steps or 
components of these policies, but they include both eligibility criteria for 
incentives and distribution calculations. The policies were analyzed for the 
following types of information: 

Bimiber of standards included to be eligible for an incentive award 
and other eligibility criteria. 

Whether standards must be exceeded or met to be eligible for an 
incentive award. 

How incentive awards are calculated including whether the formula 
was adjusted for the size of the Title II-A allocation. 



The analysis is further complicated since many States used combinations of 
each of these types of criteria. For example, some State formulas include 
awards based on meeting a number of standards and additional monies for 
exceeding standards. 

Eligibility Criteria . 

number of Standards Included and Other Eligibility Criteria. Of the 26 
States included in this portion of the analysis, the most common type of 
eligibility criteria for an incentive award was the requirement that a 
particular number of standards be met or exceeded. One important result of 
including a standard in the eligibility criteria is that, while inclusion 
alone does not effect the dollar amount of an incentive award, it does give 
the governor the ability to enq>hasize some policy initiative. 

There were no patterns as to the nuxnber of standards that werd incluJod in 
the eligibility criteria for an incentive award. The division among the 
States was as follows: 

Three States (12%) included seven of seven standards in their 
incentive eligibility criteria. 

Vine States (35%) included five of seven standards in their 
incentive eligibility criteria. 
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Two States (81) included four of seven standards in their criteria. 

One State (4%) included three of eight standards in their 
eligibility criteria (including the State's additional post- 
program standard). 

One State (4%) included two of seven standards in their incentive 
eligibility criteria. 

One State (4%) included nine of twelve standards in their criteria 
(including the State's five additional standards). 

One State (4X) requires that a positive average percentage 
deviation from the seven standards be achieved. An SDA in this 
State would not qualify for an award if it underachieved more than 
one standard by more than 20%. 

Eight States (31%) do not require a minimum number of standards to 
be eligible for an incentive award. Rather they treat each 
standard individually. Each standard may or may not be weighted 
equally. Some of these States also have additional standards. 

Core Standards . Some States required that a core set of standards be met 
or exceeded to be eligible for an incentive award. These core measures may 
match the required number of standards which must be met prior to eligibility 
for an incentive award, or the core standards may be only only a portion of 
the required number of standards with flexibility to the SDA in determining 
the additional standards to meet the required number. For example, one State 
requires that five of seven standards must be exceeded and that three of the 
five standards (the core standards) must be Adult Entered Employment, Welfare 
Entered Employment, and sx^ one youth standard. Table VI-A displays which 
standards are considered core standards which must be met or exceeded. Note 
that, in addition to specified core measures which must be met or exceeded. 
States* frequently included a minimum number of standards. Core measures tend 
to indicate which standards the governor views as primary, but they do not 
have an effect on the amount of an incentive award received. 
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Tabl0 VZ--A: 



Core Performance Meastires Included in Blisibility 

Criteria 

Vote: "^Z** indicates a core standard 



ADULT 



YOUTH 



HEASURBS 
STATES 

alabamaI/ 

COMNECTICUTi/ 

INDIANA 
lOHAl/ 

MASSACHUSETTS^/ 
MINNES0TA5/ 

MISSISSIPPI 
OiCLAHOMAi/ 
WEST VIRGINIA 



*EER WELFARE COST/EER AWP PTR EER COST/PTR 



X 
X 
X 
X 
X 

X 
X 
X 



X 
X 
X 



X 
X 
X 



X 
X 



X 
X 
X 



X 
X 



X 
X 



X 
X 



Entered Employment Rate (EER) 
Welfare Entered Employment (Welfare) 
Cost Per Entered Employment(Cost/EER) 
Average Wage at Placement (AWP) 



Positive Terminate Rate (PTR) 
Entered Employment Rate (EER) 
Cost per Positive Terminaticn 
Rate (Cost/PTR) 



l^Meet any two of the four 

i/pius two from bal-nce of the standards 

l/pius four from baxance of the standards 

l/pius either other adult measure and any one youth measure for 70% of 

incentive funds. Remaining 30% of incentive funds given if SDA meets 

additional criteria. Massachusetts has separate incentive awards system for 

service to welfare recipients. 

5/pius three from balance of the standards 

1/Plus two from the balance of the standards 
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other gligibilitv Criteria . Several States also included other 
eligibility criteria which must be met before calculating the amount of an 
incentive award. The Department of Labor has issued no formal directive on 
eligibility criteria for incentive awards but has stated at meetings that 
additional administrative standards are not appropriate. States have instead 
used administrative criteria which must be met before being eligible for an 
incentive award. The most common used additional eligibility requirement was 
one iavolving expenditure of funds. Although the law allows expenditures for 
three years from the date of appropriation, four States (Arkansas, Oklahoma, 
Texas, and Wisconsin) required that a certain percentage of the Title II-A 78% 
funds be expended. Two States, Mississippi and North Dakota, have a separate 
performance standard on expenditures. In another portion of the stmooary, 
Kansas indicated they were considering adding such a standard for FY 85. The 
percentage expenditure requirement ranged from 70% to 90%, with the State 
requiring 90% expenditures of the fxinds specifying that '•administration** be 
excluded from this requirement. In all instances. States were trying to avoid 
rewarding an SDA which was achieving standards in terms of rates yet serving a 
small ntimber of clients. 

Finally, other eligibility criteria were also mentioned. One State's 
policy provides that an SDA is not eligible for an award if there is an 
unresolved audit. Another State requires that (1) enrollment levels for WIN 
registrants and dropouts must be attained, (2) the grantee must be in 
substantial compliance with the Act as determined by the State's annual 
assessment, and (3) the grantee must submit all end-of year reports. 

Whether Standards Must be Met or Exceeded. The second type of eligibility 
issue is whether States require that standards be met or exceeded. §./ 
Frequently, State policies indicated that standards must be either '*met or 
exceeded.** In this analysis that response was considered to be **met'* since 
that was the minimum requirement. Of the 26 States included in this analysis 
12 (46%) had policies requiring that standards be exceeded before incentive 
funds could be awarded. Twelve States (46%) indicated that standards must 
only be met before awards were calculated. 1^ Of the States that used **met** 
as the criteria some had a mixed approach. One State indicated that SDAs must 
meet all and exceed five standards before distributing funds. One State's 
policy was that seven standards must be met and four standards exceeded. 

One State used a point system to calculate amount of funds and this system 
allowed variation from each standard ranging from one point for up to -15% 
deviation from the standard to five points for +10% points variance. Policies 
in one State could not be determined. 

What Constitutes Meeting a Standard . Analysis of how meeting a standard 
is defined was not very fruitful since this area was frequently not discussed 
in the policy issuances. Many States use the DOL adjustment model which 
produces a point measure for each standard* However, the DOL procedures also 
allow the use of a tolerance range adjustment within which States and SDAs do 
not have to document any further adjustment to the standard. This means that 
some States have chosen to use the range itself as the standard, e.g., ^ ^tual 
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performance within the range may qualify the SDA for an incentive award. Other 
States have elected to allow adjustments within the tolerance range but to 
still determine an actual point of measurement within the range which is used 
for the standaard. However, this becomes more complicated when viewed in 
conjunction with other aspects of the incentive policies. For example, it 
would be difficult to use the range approach if total positive variation 
(negative for the two cost standards) is used to calculate the total amount of 
an incentive award. 

Review of State policies indicates that only four States (15%) explicitly 
indicated that the standards would be a range — either the OOL tolerance range 
or a State determined range. One of these States used the standard deviation 
among all the SDAs for each measure to define acceptable performance for each 
measure . 

Distribution of Incentive funds. 

Adjustment for the Size of the Title II-A Allocation. The most common 
distribution calculation used by States in determining the amount of the 
incentive award is to adjust the amount of the potential award by the size of 
the entity's Title II-A allocation. 

Of the 26 States' policies analyzed, twelve States (50% of those 
responding) indicated that the amount of the incentive award is adjusted based 
on the grantees Title II-A allocation. Eight States (31%) indicated that no 
such adjustment is made. Six States (3 9%) have mixed approaches: one State 
adjusts three standards (the two entered employment standards and the adult 
cost standard) based on the 78% allocation; one State adjusts the available 
pool of money so that 60% of the incentive funds are based on the Title II-A 
formula and 40% of the funds are a fixed amount obtained by dividing the pool 
by the total number of SDAs in the State; one State adjusts 75% of the 
incentive award based on the Title II-A formula and 25% of the award is based 
on share units based on other criteria; one State uses 70% of the 6% incentive 
allocation adjusted for Title II-A allocation based on the State's weighted 
formula for each standard and 30% for number of additional standards met; one 
State awards a fixed amount for each standard exceeded and an adjusted amount 
(bonus award) for percent exceeded to a maximum bonus of 20% of the SDA's 78% 
grant; one State adjusts one-half of the incentive funds by the IX-A 
allocation, and one-half is awarded to top five SDAs. 

Calculation of Actual Incrjntive Amount. To calculate the amount of the 
incentive award for each SDA, one issue is whether the distribution of 
incentive funds is based on total positive variance for all standards 
(negative variance for both costs measures) or whether each standard is 
treated separately. A summary of distribution formulas for 25 States is as 
follows: 

o Five States (40%) use total positive variance (negative 
variance for cost measures) to calculate the amount of the 
incentive award • 
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o Eleven States treat each standard separately either by 

weighting each standard a given amount thereby increasing or 
decreasing its value in the award system or by awarding the 
entire pool of money for that standard for axceeding each 
standard. 

o Hine States (36X) use other approaches. Some States distribute 
a fixed amount once the SOA is deemed eligible and a formula 
amount based on other criteria normally the extent to which 
standards are exceeded. In other States, the amount of the 
award is determined based on different degrees of variance 
within rallies of performance. 

Some States divided the available incentive funds into separate pools of 
funds. This was accomplished either by dividing the funds into seven equal 
pools (one for each standard) or into seven pools based on the weight given to 
each standard. Using total variance to allocate incentive monies tends to put 
more emphasis on national policy goals than the latter method, which 
emphasizes State policy goals. The approach of using the total variance to 
calculate the amount of the incentive awards gives the most weight to positive 
achievement on the least number of standards since an SDA could exceed most 
standards to a small degree and one or two standards to a large degree and 
still receive a large award. 

Those States which award a portion of incentive funds for ranges of 
performance have in effect capped the amount of the award. For example, an 
SOA can only get **X'*% for exceeding the standard by 10% or more. There are 
also some variations on these alternatives. One State gives a base amount if 
two of their four core standards are met or exceeded and additional fixed 
amounts for each additional standard met or exceeded. 

Table VI-B indicates which standards have different weights in calculating 
the amount of the incentive award. The effect of weighting some standards 
more then others gfves these standards added value in calculating the amount 
of the incentive aw^ird. Weighted standards may show both policy priorities and 
effect the amount of the incentive award. 

States chat weighted standards individually did not always treat each 
standard separately in awarding incentive funds . Kansas , for example , 
weighted each standard to calculate eligibility but distributed funds by 
(1) awarding a minimum grant for each eligible SDA and (2) awarding the 
remainder of the award on the percentage of standards which were exceeded. 

In addition to the States in Table VI-B, which weighted individual 
standards separately, there were other variations on how to distribute funds. 
One State divided the incentive pool first into an adult and youth allocations 
with 60% of the funds for adult standards and 40% of the funds for youth 
standards. Each standard within this framework was then treated equally. 
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Table VI-B: Weights Assigned to Individual Standards 



ADULT 



YOUTH 



MEASUHBS 


*BER 


WELFARE 


COST/EER 


AWP 


PTR 


EER 


COST/PTR 


STATES 
















KANSASl^ 


6% 


6% 


6% 


10% 


6% 


6% 


6% 


KENTUCKY 


201. 


10% 


20% 


10%^/ 


10% 


20% 


10% 


MARYLAND 


20% 


25% 


25% 


10% 


5% 


5% 


10% 


MISSOURI 


20% 


15% 


15% 


15% 


15% 


10% 


10% 


NEW HAMPSHIRE 


25% 


25% 


7% 


25% 


1% 


15% 


2% 


NEW JERSEY 


20% 


25% 


20% 


10% 


5% 


10% 


10% 


TEXASi/ 


20% 


10% 


20% 


20% 


5% 


20% 


5% 



^Entered Employment Rate (EER) 
Welfare Entered Employment (Welfare) 
Cost Per Entered Employment (Cost/EER) 
Average Wage at Placement (AWP) 



Positive Terminate Rate (PTR) 
Entered Employment Rate (EER) 
Cost per Positive Termination 
Rate (Cost/PTR) 



Instate has five additional standards which make up the remaining 
weights. These weights are used for determining whether SDA was eligible 
for an incentive award not for calculating the amount of the incentive award. 
i^Meeting five of seven standards is treated as a 10% share of the 
incentive funds and PTR is included as the of the five. 

3/sDAs receive some portion of incentive funds for exceeding tne following 
standards: Adult EER, Adult Cost Standard, and AWP plus one of the 
remaining. Texas has four funding tiers based on extent to which SOAs 
exceed each performance standard. 
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Four States determine distribution by accumulating points for either the 
degree of variance or simple positive variance and then award incentive funds 
to the top SDAs (the number varies depending upon the number of SDAs in the 
States). Vote that some SDAsi which exceed all individual standards ^ may not 
receive any incentive award. 

Policies for Failure to Meet Performance Standards 

The Act requires that if a program fails to meet performance standards for 
two years, **the Governor shall isqpose a reorganization plan.** However before 
such action is taken the governor must **provide technical assistance to 
programs which do not meet performance criteria. **Z/ The Department of Labor 
has issued no instructions in writing to States on formulating sanction 
policies on failure to meet performance standards (commonly referred to as 
**sanctions** policies but here used to refer only to performance standards 
sanctions); however DOL has stated that States can not sanction SDAs on other 
than the Secretary's seven standards 

At the time of this survey only two States had plans in place. These 
approaches were quite different. One State, Nevada, concentrated on defining 
**failure**. In this case the degree of failure was the key to the imposition 
of a sanction. The policy was stated as follows: 

1. Failure to meet adult entered employment rate, youth positive 
termination rate, and welfare entered employment rate 
perfoxrmance standards by 7%. 

2. Failure to meet any two of the above by 10%. 

3. Failure to meet any one of the above by 15% plus any one of the 
remaining six standards. 

This policy did not specify action to be taken for such failure. 

The other State, Texas, focused on corrective action to be taken for 
failure to meet any given standard. It is characterized by its strong 
enqphasis on corrective action which is recommended after failure to meet any 
one standard for one year and required after failure for two consecutive 
years. The performance standards/corrective action system operates on an 
ongoing annual basis rather than the two j^^ars of the planning cycle. The 
plan offers examples of the types of corrective action Which mighh be taken 
but does not tie corrective action to the degree of failure although the plan 
does recognize that corrective action will vary by the degree of failure. 

Tho two policies in place show the diffi ulty of - trying to develop a 
comprehensive approach to imposing . motions. In neither of these States is 
there a direct link to incentive and sanction policies. In Ne^'ada, SDAs must 
meet all seven standards to be eligible for an incentive award, but failure to 
meet one standard by a small amount does not mean that sanctions will be 
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imposed. In other words there is a gray area where neither an incentive award 
would be made nor any sanction required. It may be that this area also 
qualifies an SDA for technical assistance but that is not indicated in the 
policy. Texas treats incentives and sanctions as completely separate policy 
matters. An SDA ca>. receive an incentive award at the same time they also 
fall under corrective action. 
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SBCnOV II: YOUTH EMPLOYMBHI COHPBTEHCIES AHD TEBMISATIOH MEASURES 



The questions in this section detx with the status of youth employment 
cooqietencies at the SDA level and St&te policies and practices in developing 
competency systems and adjusting the current youth performance measures. 

At Section 106(b)(2) the Act specifies that the Secretary ''shall designate 
factors for evaluating the performance of youth programs which, in addition to 
appropriate utilization of the factors described for adults shall include the 
attainment of recognized enq>loyxQent cooq^etencies recognized by the private 

industry council The Department of Labor indirectly included the 

attainment of employment conqpetencies for youth in the definition of two of 
the national performance measures — positive termination rate and cost per 
positive termination. However, DOL did not include a reporting item for the 
attainment of eo^loyment conqpetencies for youth on the JASR. The decision on 
how to incorporate the attainment of yo^ith employment competencies (or the 
lack of such systems at the local level) was left up to the Governor. 

The N6A survey indicates that 60% of the SOAs in the forty-nine States 
responding to this part of the survey have youth employment competency systems 
actively serving youth. States reported that an additional 114 SOAs will have 
operational systems by July 1, 1985, which would bring the percentage to 77% 
of the 579 SOAs in responding States. 

Status of Youth Employment Competencies Systems 

Table VII indicates the number of SDAs in a State who have ''developed 
youth eiiq)loyment competency systems'* or the number of SOAs who plan to have 
such systems by July 1, 1985. ''Developed youth employment competency systems" 
was defined as one which h^a been approved by the PIC and is actively serving 
youth within the competencies approach. The responses to the question about 
the number of SDAs with operational youth employment conq)etency systems does 
not correspond exactly with other recent surveys on this subject. The 
Hational Alliance of Business (NAB) and the National Association of Private 
Industry Councils (NAPIC) did an informal tdephone survey of States in April 
on this issue and obtained somewhat different results. The portion of the GAO 
survey on the developmental status of youth employment competencies showed a 
higher total number of SDAs with competency systems in place. However, the 
GAO survey questions were asked directly of SDAs whereas the NGA questions 
were directed to the States, which may not have the most current information 
about local systems. 
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SDA8 Development of Youth Employment Competencies Categories, 

In the forty-three States responding to this question, 98% of the total 
SDAs were represented* Of these SDAs repres^mted by their States, the survey 
revealed that pre-employment competencies are still the most common categories 
of competencies offered with work maturity a close second. In some cases the 
two categories are combined by SDAs, but this assessment considered them as 
separate categories* The breakouts by category of youth employment 
con^etencies are as follows: 



o Pre*-enqployment — 281 SDAs (57%), 

o Woric Maturity~267 SDAs (54%), 

o Basic Education — 132 SDAs (37%), 

o Specific Occupational Competencies — 153 (31%) • 



Fourteen States indicate! that at least one SDA in the State had conq>etencies 
systems in place for all types of competencies. 

Six States could not provide information broken down by the four types of 
competencies: pre-employment, work maturity, basic education, and specific 
national competencies. This reflects the fact that there is no requirement 
fur such a breakout and since most States do not collect data by specific type 
of competency attained. 



States noted that slightly less than 1/3 of the SDAs had occupational 
specific competencies, ^^/enty-eight States would have one SDA with a youth 
employment competency system offering occupational competencies. 



State Level Youth Competency Reporting. 



There is no national requirement for reporting the attainment of youth 
enq)loyment competencies to either the Federal or Stzte level, but States have 
begun collecting data in this area on their own. Of the forty-eight States 
(including those with planned reporting for PY 85) responding to the question 
on number of reporting categories for the attainment of youth employment 
competencies, most States currently have a reporting system that reports the 
attainment of youth emplojnnent competencies. Most States use one reporting 
category. W The following specific information was reported by States. 

41 States (85%) currently have a reporting system that reports 
the attainment of youth employment competencies from their 
SDAs. 
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6 additional States (13%) plan to institute such reporting in FY 
85 which would bring the total to 98% the middle of calendar 
year 1986* 

1 8ingIe«*State SDA was not sure about reporting « 

The majority of the forty-eight responding States use one reporting 
category, '*attainsient of youth employment competencie.^**: 

c Thirty-four States (71%) either use or plan to use one 

category, attained youth employment competency,** for reporting 

from the SDAs« Two of these States plan to use multiple 
reporting categories in FY 85 « 

o Twelve States (25%) plaa to use multiple reporting categories 
(see table for details^* 

o Two States (4%) were undecided about reporting categories 
(undecided on the table). 

See Table VTI for specific responses. 

Categories Required Prior to Certification of the Attainment of Youth 
Employment Competencies by the State. 

Of the thirty-four States that use one i :;porting item, **attained youth 
employment competencies,** only tour States (1L%) indicated that all areas of 
employment conq>etency in which a youth was identified as deficient at time of 
assessment must be attained before the attainment of youth employment 
competency could be reported. UL^ These States were Illinois, Nevada, North 
Dakota, and Rhode Island* One additional State noted that this decision 
varied from SDA to SDA, and there was no State policy on this matter. 

Reporting of Youth Employment Competency on the JTPA Annual Status Rapotfc 

The JASR cu:Tently does not have a discrete reporting item for the 
attainment of yourh es^loyment competencies. Rather it is reported as part of 
**all other terminations,** Item I.B.3. 

Of foirty-three states that responded to the question of reporting the 
attainment of youth employment conqpetencies to the Federal level, 30% of the 
respondents were not sure where this data element was reported on the JASR. 

7 States (16%) indicated that attainment of youth employment 
competencies is reported in item I.B.2, Youth Employability 
Enhancement 
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30 States (70%) indicated that attainment of youth employment 
competencies is reported in item I.B.3, All Other Terminations 

3 States (7%) stated that it is not reported on the JASR 

3 States (7%) reported a separate reporting category apparently 
on their own State version of the JASR 

The inconsistency in responses raises some concern about the validity of 
current JASR reporting and highlights the need for a separate dat& time to 
report youth employment conqpetency attainment. 

Administrative Requirements for Reporting the Attainment of Youth Employme\it 
Competencies to the State. 

Forty-six States responded to this portion of the questionnaire. The 
assessment wanted to determine whether States had assigned any additional 
adminis^ratiV2 criteria which must be met by the SDA before the attainment of 
a youth esployment competency can be reported to the State, 

The criteria established by States do not appear to restrict local 
discretion envisioned under the legislation. 

o Twenty-four States (52%) which had operational youth employment 
competencies systems had no such administrative requirements. 

o Twenty-two Jtates (48%) did have some administrative 
requirements. 

Table VIII indicates the administrative criteria which SDAs must meet 
before the attainment of youth employment competency can be reported to the 
States. 

o Sixteen States (72% of those with additional requirements) 
noted that the competency must be certified in writing. 

o Nineteen States (86% of those with additional criteria) said 
the cos^etency rust be measurable. 

o Fourteen States (64% of those with additional criteria) stated 
that the competency must be attained through participation in 
JTPA. 

o Fourteen States (64% of those with additional criteria) had 
••other** criteria. 

Responses in the **other** category most commonly included a requirement for 
pre-and post-testing or some type of assessment of deficiencies (eight 
States). Additional responses included: cooqietencies must be employment- 
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related, documentation required, must have a program design statement, 
competency must be in one of the four basic areas, and a policy on content and 
proeedur«)s . 



It is important to note chat all of the criteria are "process" oriented and 
do not appear to encroach upon PiC discretion about the selection of 
recognized conq>etencies appropriate for the local labor market. 



lABLB VIII: ADOITIOHAL ADMIVISTRAIIVB REQUIBEHENTS 
FOR YOUTH EMPLOYHEHT COMPETENCIES 
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z 
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NORTH CAROLINA 
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NORTH DAKOTA 
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z 
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14 
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Review and Approval of the Youth Employment Conmetency System 



The role of the State in the youth employment competency system has been 
confined to two major areas: 

o Providing technical assistance in developing youth employment 
competencies, and 

o Determining the effect of youth employment competencies on 
performance standards 

To accomplish these responsibilities (primarily the latter one), States 
have recognized the need for reporting attainment of youth employment 
competencies as noted earlier. To ensure a meaningful employment competency 
approach, some States have developed additional criteria before this program 
outcome can be reported to the State. Under the statute. States do not have 
•^approval** authority for each specific competency developed in an SDA; only 
the PIC does. This survey indicates that States understand their role. 

Some States have required SDAs to describe their competencies in the Job 
Training Plan (JTP). Table XX indicates the different States* approach to the 
review and/or approval process. Thirty-four States answered this question 
about the role of the State in re lewing and/or approving youth employment 
competencies systems established by the PICs. 

Fifteen States (4A%) indicated that they reviewed and approved youth 
employment competencies systems. Fourteen States (41%) reviewed competencies 
only. Five States (15%) do not review and approve any part of the 
implementation of youth employment competencies by local programs. 

Some States that indicat&d they did not have specific criteria for 
employment competency systems stated that they do in fact "review and approve** 
SOAs* youth employment conqpetency systems. It is unclear whether this is an 
informal review as part of the review of the Job Training Plan (JTP) or a more 
explicit approval process. Since the statute in Section 106 (b)(2) jlearly 
lays out the conditions for rejection of the JTP and this does not include the 
disapproval of the JTP due to the absence of a youth employment competency 
system, the ^•approval** may be of a general review nature. 

Adjustments of Youth Positive Termination and Cost Per Positive Termination to 
Account for the Presence or Absence of a Youth Employment Competency f ystem 

Of forty-six States responding to this question, only twelve (26%) stated 
that they adjusted either of these two standards for the presence or absence 
of youth employment coiq>etencies at the local level in PY 84. It may be that 
only those States that made adjustments at the beginning of PY 84 for planning 
purposes responded positively to this question and that others will do so as 
final end-of-yeat* standards based on actual performance are calculated. One 
State, Georgia, has developed a methodology to make such an adjustment based 
on the type of youth programs actually ir place. 
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For PY 84 both the national standards for positive termination rate and 
cost per positive termination and the 005^ adjustment model for these standards 
were developed from CETA data sinc« no JTPA data were available* This meant 
that youth employment competencies could not be included in the data base used 
to calculate these standards. Instead, DOL used CRTA Title IV data on 
-completed program objective- to substitute for attainment of youth employment 
competencies as a reportable program outcome. Governors (States) were given 
the responsibility to adjust the two affected youth performance standards to 
accotmt for whether youth enqployment competency systems were actually in place 
and serving youth. 

This was envisioned to give States the latitude to adjust the positive 
termination rate and the cost per positive termination in various ways 
depending on the status of competency systems in the SOAiu For example, the 
model would predict too high a positive termination rate if an SDA had no 
cosqpetency systems in place and only reported placement and other outcomes. 
SDAs with significant non-enq;>loyment related youth programs would also be 
unable to meet the positive termination rate (PTR). On the other hand* SOAs 
who had youth employment conqpetency systems actively operating and terminating 
youth would presumably be able to achieve a higher positive termination rate. 
In both instances* States might need to adjust the youth performance standards. 

The intention at the national level was for States to adjust the PTR 
downward if competency systems were not "suf flciently developed-l?./ or if 
SDAs chose to operate primarily employment directed programs. During the 
transition year non-employii;^nt program activities accounted for only 8% of all 
youth terminations. 

For PY 85, nineteen States (41%) indicated that they were either planning 
to make such an adjustment or were unsure at this time. The problem with the 
model and the national standards for these two youth measures continues in PY 
85 despite the fact that the DOL adjustment model used actual JTPA transition 
year data. This is because the attainment of youth employment competencies is 
not a Federal reporting item on the JASR and therefore not a part of the data 
used to calculate the adjustment model. il^ The national numeric standards 
did not change for PY 35. 

During the transition year the nation as a whole exceeded the national 
youth entered employment rate of 41% by 14 percentage points. However, the 
nation failed to meet the national standard for positive termination rate of 
82% by 19 percentage points presumably because attainment of youth employment 
conqietencies (if any, due to the start-up of the competency approach) was 
included in other negative outcomes on the JASR. 

WeiKhtinK of Youth Standards in Making Incentive Awards 

Our assessment shows that most States did not attempt to weight youth 
positive termination and youth cost per positive tenolnation standards 
differently than other standards. Seventeen States (52%) of thirty-three^ 
States responding gave all seven standards equal weight. Only a few States ' 
weighted the positive termination rate and cost per positive termination 
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standards significantly less than the entered -employment- rate for youth* W 
This may have been because entered employment was viewed as the primary goal 
in these States rather than due to an attempt to lessen the impact of newly 
emerging youth w ^oyment competency systems* 

This issue is closely tied to whether States adjusted (or planned to 
adjust) the positive termination rate standard and the cost per positive 
termination standard. Since States could give different policy emphasis to 
certain outcomes by weighting individual standards differently, one option to 
minimize the effect of the two standards influenced by the lack of youth 
employment con5)etency data was to weight these two standards less than the 
entered employment rate. 
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SBCnOH III: P0ST-PR06RAH FOLLOW-UP 



This section was included in our assessment because of the emphasis given 
to post-program performance standards under the legislation • 

LeKislative B ackground 

Section 106(a)(1) of the Job Training Partnership Act (JTPA) states that: 

(1) it is essential that criteria for measuring the return on 
investment be developed; and 

(2) the basic return on the investment is to be measured by the 
increased employment and earnings of participants and the 
redaction in welfare dependency. 

The Act suggests appropriate performance measures such as retention in 
unsubsidized employment and the increase in earnings and reduction in the 
niunber of individuals and families receiving cash assistance welfare 
payments. Further, in Section 106(b) (3) (A and B), the Secretary of Labor is 
directed in establishing performance standards to select "the base period 
prior to program participation that will be used** and "a representative time 
period after termination from the program that is a reasonable indicator of 

post-program earnings •* This clear legislative mandate indicates the 

importance of post-program measures in assessing the success of JTPA. 
Post-program measurements of the effect of the program on participants have 
long been accepted as valuable information for all levels of program managers 
in employment and training programs. 

In general, this assessment shows that States and local progtam operators 

have established post-program management systems that include follow-up 
information. 

Status of Follow-up Systems 

Thirty-nine States (80%) will have operational follow-up systems in place 
by the beginning of P!f 86 according to responses to this questionnaire. See 
Table X for specific responses. In addition, the assessment of the status of 
JTPA post-program follow-up revealed that: 

0 Twenty-eight States (57% of those responding) have follow-up 
systems in place and are collecting data. Eleven States (22%) 
have been doing some follow-up since the inception of JTPA or 
before. 

o Eleven additional States (22%) plan to have such systeios by the 
beginning of PY 86 (July 1, 1986). See fable X or XI for 
specific responses. 
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o Seven States (14X) indicated that they had no ^^ans fcr a 
follow-up system. 

o Three States (6%) indicated they would wait for further 
information from the Department of Labor about whether a 
follow-up system would be required before taking any action. 

States were asked whether they performed follow-up at the State level or 
required the SDA to do so. This assessment does not include any information 
about the SDAs who have their own post-program follow-up systems. The 
assessment, then, cannot express the full extent of follow-up activity in 
place throughout the JTPA program. 

Principal Means of Post-Program Data Collection 

The most common type of follow-up appears to be that required by States to 
be undertaken at the SDA level. See Table XI and ZII for specific responses. 

o Sixteen States (57% of those States wh;ch have operational 
follow-up systems) require SDA-level follow-up. 

o Participant follow-up was the most common method of follow-up 
required at the local level although eleven States indicated 
that employer contact was also usedi^/ (Soe Table XII) « 
Responses did not indicate whether SDAs are required to do both 
eziq>loyer and participant contact or whether they have the 
option of doing either. 

o Five States (18%) indicated that the SDA could choose its own 
methodology. 

o Further, it appears that seventeen States require some type of 
local follow-tq> without specifying who must be contacted. 

o Only two States (7%) use unemployment insurance records as 
their sole means of follow-up. 

The size of the JTPA allocation does not appear to be a strong factor in 
whether States have implemented a follow-up system. Thirteen of the 
twenty-five States with the largest allocations have implemented some type of 
follow-up. On the other hand, nine of the sixteen States with an allocation 
of less than $10 million have operational follow-up systems of some type. Six 
of these States are single-State SDAs. 

Regard^ag the accessing of AFDC. records, some States indicated that they 
plan to access such records, but the survey does not indicate how many 
actually have implemented such approaches. Nonetheless, fifteen States (31%) 
indicated that they are either currently collecting AFDC data or plan to 
collect information from AFDC records either by a statewide sample of AFDC 
records or by using an SDA-specific fii^pvo^ch* In addition, three States 
require their SDAs to access local welfare records (See Table XII for further 
information). 
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ZABLS Z: Stattis of Follo%i-u7 





Operational 
Follow-up 
Svstttns 


Planned 
Systems 


No Current 
Plans for 
follow-uo 


Waiting 
fO" DOL 


ALABAM 


Z 






ALASKA 


Z 








AfilZOHA 






Z 




ARKANSAS 




Z(7/85) 






CALIFORNIA 




Z(7/36) 






COLORADO 






X 




CONNECTICUT 






X 




DELAWARE 


z 








FLORIDA 


z 








GEORGIA 


z 








HAWAII 


z 








IDAHO 






xi/ 




ILLINOIS 








Z 


INDIANA 






z 




IOWA 


z 






— — — 


KANSAS 


z 








KENTUCnf 








z 


LOUISIANA 


z 








MAINE 










MARYLAND 


z 








MASSACHUSETTS 


z 








MICHIGAN 


z 








MINNESOTA 






z 




MISSISSIPPI 


z 








MISSOURI 


z 









ERIC 



37 



43 



TABLE Z: Statu* of Follow-up (Cont'd) 



Oporational no Current Waitins 

Pollot?-up Planned Plans tor for DOL 

Systems Systems follo*»-uo Guidance 



KONTAHA Z 



HEBRASXA 


Z 






HEVAOA 




Z(7/85) 




HEW HAMPSHIRE 




Z(10/85} 




HEW JERSEY 




Z( 7/85)1'' 




HEW MEXICO 






Z 


HEW YORK 




Z(PY8S) 





HORTri CAROLIHA Z( 10/85) 



HORTH DAKOTA X(7/85) 



OHIO 




Z(l/86) 




OKLAHOMA 


Z 






OREGOH Z 



PENHSYLVAHIA 



RHODE ISLAHO 

SOUTH carolih;. 

SOUTH DAKOTA 



TENNESSEE 



TRZAS 



UTAH Z 


VERHOHT 


Z 








VIRGIHIA 




Z(7/8r) 






UASHIHGTOH 


z 








UISCOHSIH 








Z 


D.C. 




Z(10/85) 






TOTALS 


28 


11 


7 


3 
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Timing of Post-ProKram Data Collection 



Of those twenty-six States that currently use some form of participant or 
employer follow-up (either state-administered or SDA-administered follow-up), 
the following post-program time periods are used: 

o The vast majority of States (20 or 77%) contact the participant 
and/or the employer at thirteen weeks after the participant 
terminates •lii^ 

o One State (4%) does twenty-six week follow-up and four States 
(15%) use a longer time period in addition to thirteen weeks* 

o Two States (8%) use thirty days as the required follow-up 
period, 

o Three states (12X) contact participants sometime during the 
quarter after termination (**other** on Table XI) but not at a 
set time period. 

Terminees and Age Groups Tracked During the Post-Program Period 

Twenty-nine States responded to the portion of the questionnaire 
regarding who was tracked in their follow-up system and what information was 
obtained. Of that number: 

o Seventeen States (59%) responded that all terminees were 
included in the post-program system. IZ^ 

o Eleven States (38%) responded that only terminees who entered 
employment were tracked. 

Of the twenty-nine States that do some follow-up: 

o Twenty-five States (86%) follow-up on all age groups within the 
terminee population (adults, youth, and adult welfare 
recipients) . 

o Four (14%) States reported that only adults were included in 
their follow-up systems. 18/ 

Information Collected at the Time of Follow-up 

Of the twenty-six States which iu:e some participant contact and follow-up 
on both adults and youth, twenir.y-two (85%) ask the same questions of all 
client groups.- These States all obtain post-program information on hot^rly 
wage and current employment status. ii^ (Table XIII provides specific 
information about what data is collected) . Some States also provided 
information on pre-program information obtained on these participants by 
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TABLE nil: 



Inf^nutlon Coll#ct«d for all Adult and Youth Tantlnaas 
(Participant or Ivployar Contact) 
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mabchins the client record with their management information system. The 
survey was not able to collect consistent information on: 'Tiours worked** on 
the current job, **weekc worked** during the follow-up period, or •*hours worked** 
on the current job, but soioa States did provide this information. 

Post-Program Performance Measures How in Use by States 

Two States, Kansas and Washington, implemented formal post-program 
performance standards for PY 84. Kansas used two post-program measures both 
related to post-program eamings20/ (along with three termination-based 
standards in addition to the Secretary's seven standards). This State did not 
use their post-program standards part of their incentive system. The two 
post-program standards each had a weight of 14% in determining whether 
standards had been met, giving them the most emphasis of any standards used by 
the State. 

Washington State established an employment status measure (whether the 
participant was employed at any job at time of follow-up). This standard was 
part of the incentive award system for PY 84. 

Significant efforts are underway in the States to collect data to 
establish post-program performance data which could be used to establish 
future numeric standards. Eight of the twenty-eight States (29%) currently 
collecting data indicated that they were considering post-program measures for 
the near future. These Staters are: Alaska, Georgia, Massachusetts, 
Mississippi, Nebraska, Rhode Island, Texas, Vermont. 

These States were not positive about the types of measures they would 
establish, but they did indicate that their plans were to establish measures 
in the following areas: earnings increase (six States), job retention in the 
same job (six States), employment status in any job (five States), welfare 
status (five States), percent of time employed in weeks (one State). 
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SBCTZOH IV: ADDITIOHiL PERFOBHAVCS SIAHDARDS 



This section assessed State activity in establishing performance standards 
in areas other than Title II-A^ specifically in the areas of older worker (3%) 
programs, state education coordination grants (8%) programs. Title III, and 
equity standards for particular target groups. The Act at Section 106(g) 
provides that ••the Secretary shall prescribe performance standards under Title 
III based on placement and retention in unsubsidized employment.** No 
standards are explicitly required by the Act for the 3% or the 8% programs. 
However, at Section 203(b)(3) the Act specifies that •'recipients of payments 
made under the program of aid to families with dependent childir^* under a 
State plan approved under part A of Title IV of the Social Security Act who 
are required to, or have, registered under section 402(a)(19) of that Act and 
eligible school dropouts shall be served on an equitable basis, taking into 
account their proportion of economically disadvantaged persons sixteen years 
of age or over in the area.** Because these two groups were given special 
consideration for service in the Act, some States established specific 
performance standards for these target groups; other States chose to use 
ongoing moaitoring to determine SDA compliance against equity of service for 
these two target groups. Whore standards were used, the standards were of two 
kinds: separate entered enqjloyment standards or equity of service standards. 
•'Equity of service** standards generally set the numeric goal for how many of a 
particular demographic group must be served in relation to their incidence in 
the eligible population. Under JTPA some States have included those groups 
with recognized barriers to employment in the State in separate equity of 
service standards under provisions of the Act at section 202(b)(3)(B) which 
specif! s that incentive awards may be made for serving hard-to-serve 
individuals. 

The first question in this section dealt with the number of States that 
used additional standards for equity of service or to assess other specific 
state goals such as economic development linkages. 



Establis hment of Performance Standards for the Following Types of Target 
Groups or State Initiatives: Public Assistance Recipients, School Dropouts. 
Economic Development Linkages > or Other Programs 

While eighteen States (37%) either used such standards in PY 84 or plan to 
do so in PY 85, there is no significant activity in setting standards for 
••equity of service^* for any one particular group. This may be because equity 
of service for target groups is viewed in the monitoring context rather than 
as a performance standards function. The survey did not specifically question 
whether entered employment standards were used or whether equity standards 
were used. It is unclear whether States included these standards in their 
incentive award systems since answers to that question were inconsistent with 
responses cn the use of the 6% funds (see Table V). Of course, there are 
methods other than incentive awards to encourage service to the hard-to-serve. 



Seven States (14%) indicated they had standards in PY 84 for public 
assistance recipients. Nine States (18%) noted that they planned standards in 
this area for PY 85. 

Four States indicated they had standards for school dropouts in PY 84, and 
six States planned standards in PY 85. 

Other types of additional standards mentioned by the responding States 
included (mentioned by one State per each response unless otherwise indicated): 
linkage and coordination standard (PY 85), cos- per-entered-employment (youth); 
equity standards for target groups such as ex-offenders (two), veterans, handi- 
capped, displaced homemakers, percent female, percent minority; maintain OJT 
levels, youth enrollments, indirect placement rate. 



Bst'ablishment of Additional Employment Standards for 3% Programs, 8% Programs > 
or Specific Target Groups 

It appears that States have put less emphasis on establishing performance 
standards for State set-aside programs probably due to the diverse goals of 
these programs throughout the States. This may also be due to concentrating 
staff resources on starting up these programs, but there is no significant 
increase from PY 84 to PY 85 in the number qf States planning to have standards 
for the 3% or 8% programs. However, subsequent to this survey, NGA has learned 
that a number of States who did not respond that they had standards for the 3% 
and the 8% programs have in tact implemented standards for these programs. 

In PY 84, seven States (14%) established entered employment standards for 
older worker programs, and two additional States indicated they would do so in 
PY 85. Five States used cost per entered employment standards for their older 
worker programs in PY 84, and two additional States planned to use such 
standards in PY 85. 

For PY 85, seven (14%) States established such standards for the 8% set- 
aside as well, and four additional States planned to do so fc^.* PY 85. Two of 
these additional States indicated they would use goals rather than standards. 
Five States noted that they used cost per entered employment for PY 85 for at 
least a portion of the State education set-aside; one additional State antici- 
pated adding such a standard in PY 85; two States planned to add goals. 

Five States (10%) noted that they would have some type of entered employ- 
ment standards for groups such as handicapped participants, ex-of fenders* and 
other hard-to-serve serve groups. 
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Establishment of the Required Title III Measure of Entered Employment for PY 84 



The Act requires the Secretary of Labor to establish performance 
standards for Title III "'based on placement and retention in unsubsidized 
employment ,-21/ since there was no experience with these programs 
nationally to establish national standards and since these programs were 
expected to vary considerably from State to State depending upon the 
conditions of the labor markets within the States, DDL gave much latitude to 
the States in establishing Title III standards within the State. In 
Performance Standards Issuance I-PY 84 dol required governors to establish an 
enterv'id employment rate standard for each of their Title III programs (formula 
funded only). Governors were also encouraged to "^establish goals for the cost 
per entered employaient, which take into consideration the Title III program 
design, participant characteristics, and other factors deemed appropriate by 
the Goven'iOc-.'* 

States were asked to describe their data sources and rationale for 
establishing the numeric standard for the entered employment measure. States 
used a combination of data sources to establish standards. Frequently, States 
indicated that they used a combination of Title II-A approaches (either the 
national standard, the Title II-A adjustment methodology, or using Title II-A 
as a floor) in conjunction with previous Title III experience, of the 46 
States who responded to this question: 

o Four States (9%) used the Title II-A entered employment standard as 
a base or floor as the only approach to setting a standard. This 
means that the Title III entered emt-loyment rate could be set no 
lower than the Title II-A entered employment rate. 

o Bight States (17%) specified that they used the Title II-A national 
standard for entered employment (55%). 

Five States (11%) used the Title II-A adjustment model to arrive at 
Title III standards. 

Bight States (17%) indicated that they used some form of the Title 
II-A entered employment standard but did not indicate what this 
meant. Probably one jf the three methods discussed above was used. 

o Four States (9%) noted that they used CETA data (one State 
indicating Title VII) to establish these standards, but they did 
not indicate how this was accomplished. 

o Three States (7%) indicated that they used Transition Year Title 
III experience exclusively to establish the standards. Four 
additional States used this data in conjunction with Title II-A 
experience to inform their decisions. 



46 



54 



o The remaining 14 States (30%) used **other'* methods, which ranged 
from a use of labor toarket information, negotiated standards, and, 
as one State honestly stated, '*guessing.^ 

In some cases, states indicated that Title III funds were allocated by 
formula or roquest-for-proposals (RFP) to SDAs and that SDAs were then 
required to meet their previously established Title II-A entered employment 
standard which might include other negotiations. Several States indicated 
that while Transition Year Title III information was used in conjunction with 
Title II-A data, {\b Title III entered employment rates were negotiated as 
part of a performance-based contract* All States appeared to use a variety of 
data sources, but fairly standard approaches to setting the numeric values of 
the standards. One State, New York, attempted a regression model using their 
own data from their projects but did not finally use it for PY 84 to set 
standards. The modeling approach is still under consideration for PY 85* 

Title III activities include such things as job search assistance, 
customized training, OJT, rapid response teams to plant closings and a host of 
'ler activities all of which may not lend themselves to entered employment as 
wue only outcome standard. One f^hate , Washington , mentioned an approach to 
setting standards that acknowledf 5 the different types of program strategies 
or interventions used in Title ^ .i programs. This methodology used weighted 
averages from historical rates for various program activities adjusted by one 
client characteristic (UI claimant), but it was unclear whether this proposal 
was actually being used or was still in draft. One State responded that they 
did not believe an entered employment standard was meaningful but had 
established such a standard in response to the Federal mandate. 

Establishment of Additional Measures for Title III for PY 84 

Twenty-seven States (56% of the 48 responding) had established some 
additional performance measures for Title III beyond the entered-employment 
rate (see Table XIV for specific inforration) • Performance Standards Issuance 
I-PY 84 **encouraged** States to establish cost per entered employment standards 
and 27 States (57% of those responding) established at least that measure. 

o Twenty-four States (50%) will establish a cost per entered 
employment standard by PY 85. 

o Thirteen States (27%) established some type of wage measure. Some 
indicated that they used an average wage at placement measure. 
Others used wage as a percentage of the dislocation wage, or 
average wag^a decline, as a measure. 

o Ten States (21%) used both a cost measure and some additional 

measure — a wage measure of some type in all cases except one. 

Washington State proposed a standard on percent of placements in 
new or upgraded occupation. 



Tabls ZIV . 
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o One State, Washington, used employment retention as a measure — 
defined as the number of participants employed 13 weeks after 
termination. The retention rate was set at 70% — higher than the 
retention rate the State used for Title II-A since the State 
believed the Title III population had a strong work history and 
demonstrated attachment to the labor force. 

o Washington State also has proposed a standard on percentage of 
placements in new or upgraded occupations. 22/ while the proposal 
was still in draft form when sent to NGA it reflects Washington's 
basic premise that the dislocated workers have been employed in 
industries and/or occupation for which there is diminishing need or 
dmnand. 

Establishment of the Title III Measure of Entered Employment for PY 85 

Most States planned to use the same data sources for PY 85 as for PY 84 
plus any additional PY 84 Title III \nformation available at the time. Only 
two States planned to add a Title III standard for PY 85. lleither of these 
States had additional Title III standards (beyond entered employment rat^) for 
PY 84. 
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FOOTNOTES 



1/ Job Training Partnership Act, Section 106(e)* 

2/ Performance s;:ahdards Issuance I-PY 84, January 31, 1984; pg« 8. 
Guide for Setting JTPA Title II-A Performance Standards , March, 
1985, pg. A14-A15. 

3/ Job draining Partnership Act, Section 202(b(3)(B). 

4/ Performance Standards Issuance I-PY 84, January 31, 1984; pg. 8. 

5/ The Act at Section 202 (b)(3)(B) states that awards should bo made 
based on the degree by which the se^^ice delivery are?, -^xce^ds its 
standards. 

6/ IBID 

7/ JTPA, Section 106 (h)(1). 

8/ There is no written policy guidelines on this issue, but DOL 
officials have stated that **in their judgment it would not be 
acceptable, in accordance with Section 106 (h)(1) cf the Act.** 

9/ Performance Standards Issuance Number 3-84, October, 7, 1984. 

10/ One reporting item for the attainment of youth employment 
competencies was recommended by the JTPA Performance Standards 
Advisory Committee. 

11/ This question was included because at one time the Department of 
Labor war considering requiring that all areas of employment in 
which a youth was identificad as deficient must be attained through 
participation in JTPA before an SDA could take credit for the 
attainment of a youth employment competency. 

12/ Performance Standards Issuance Number 3-84, October 7, 1984. 
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13/ The key issue in deciding what, if any, adjustments to make was, in 
fact, the types of youth programs operated by an SDA. One of DOL's 

contractors for performance standards, SRI, showed this difference 

graphically in an analysis of information from the Job Training 
Longitudinal Survey* 

Positive-Termination Rates 

Without Attainment Witu Attainment 

of Employment of Employment 

Competency Competency 

Youth Characteristics 
School Status 

In-School 54X 71% 
Out-of-school 

Dropout 63% 651 

Graduates 75X 77% 

Age 

14-18 60% 70% 

19-21 71% 73% 



14/ See Table VI in Section II for information from incentive 
policies on this issue. In addition to the States on that 
table, New Hampshire and Florida reported weighting the PTR 
less than the EER for youth. 

15/ Participant follow-up was proposed by OOL as the standard 
follow-up system in the Federal Register Notice of April 26, 
1983, and was also recommended by the JTPA Performance 
Standards Advisory Committee in June, 1985. 

16/ The JTPA Performance Standards Advisory Committee recommended 
13 weeks as the appropriate follow-up time period. 

x7 / The JTPA Performance Standards Advisory Committee recommended 
contacting all terminees. 

18/ The JTPA Performance Standards Advisory Committee recommended 
in June, 1985, that follow-up be conducted for all adult 
terminees . 
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19/ The recotnmendations in June, 1985 of the JTPA Performance 
Standards Advisory Committee could be implemented by twenty-six 
States from the information they currently collect. 

The recommended post-program measures were: employment status 
during the 13th week; employment intensity — the number of weeks 
employed during the follow-up period; and average earnings 
during the 13th week (for those employed). 

20/ The two post-program measures and stuidards were: 

Earnings Increase (Adult) - The pr rcentage that the average 
earnings of JTPA participants during the six months subsequent 
to the quarter of termination exceeds the average earnings of a 
control group during the same period of time: 12% 

Bamings Gain per Dollar Expended (Adult) - The difference 
between the average earnings of JTPA participants and that of a 
control group (during the six months after the quarter in which 
the participants terminated) as a percentage of the average 
cost per adult termination. (Note: The cost per terminrtion 
is the total expenditures for adults divided by the number of 
adults who terminated): 12% 

21/ JTPA, Section 106(g). 

22/ Percentage of Placement in New and Upgraded Occupations is 
defined in the following formula: 

Number of participants employed at the end of 13 
weeks after termination from the program who are in 
an occupation different from the one from which they 

were displaced. 

Number of participants employed at the end of 13 
weeks after termination from the program. 
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Recommendations to the DepartBent of Labor 



The analysis of the State Implementation of Performance Standards under 
the Job Training Partnership Act revealed a number of concerns which VGA 
believes must be brought to the attention of the Department of Labor for 
resolution. These issues are confined to areas where N6A believes that action 
on the part of the Department can result in significant improvements in the 
performance standards system. 

1. It is recommended that the Department of Labor standardize the 
definitions of data element used in the DOL adjustment methodology 
and that related reporting problems be .addressed. 

2. It is recommended that DOL formally endorse the ui;e of 6X funds for 
preventative technical assistance. 

3. It is recommended that DOL add a data item on the JTPA Annual 
Status Report for the attainment of youth employment competencies. 

A. It is recommended that DOL adopt the recommendations of the jTPA 
Performance Standards Advisory Committee regarding beginning 
post-program data collection for Title II-A by the beginning of PY 
86. 

5. It is recommended that DOL provide substantial technical assistance 
to state and local programs technical assistance to implement 
post-program follow-up. 

6. It is recommended that DOL provide assistance to States in 
developing performance standards for state set-aside programs. 
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